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Remarks 

Qaims 1-27 are pending in the application. Qaims 1-27 are rejected. Claims 1, 3, 
5, 17-18, 20-23, and 27 are amended. The Specification is amended. No new 
subject matter is added. The rejections are respectfully traversed. 

I 

The Specification is amended to overcome the Examiner's objections. 

Claims 3, 5, 17, 18, 20, and 22-27 are rejected under 35 U.S.C. 1 12, second 
paragraph as being indefinite. 

Claims 3, 5, 17, 18, 20, 22, 23 and 27 are amended to overcome the rejections. 
Claim 1 is amended to more distinctly claim the invention. 

Qaims 1-3, 6-15, 19, 21, and 27 are rejected under 35 U.S.C. 102(e) as being 

anticipated by Abdel-Mottaleb (U.S. 6,424,789 - "Abdel"). 

I 

The invention is frame based. Each frame of an input video has an associated input 
frame play time, and the input video has a total input video play time that is a sum 
of the input frame play times of all of the input frames. Each of the input frames 
of the input ^ndeo is dassified according to a selected content characteristic. An 
output frame play time is allocated to each of the plurality of input frames based on 
the content characteristic of each of the input frames to generate a plurality of 
output frames. 
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Abdel measures differences between frames in a shot to classify the shot. The 
invention looks at only one frame at a jdme to classify the frame. 

To those of ordinary skill in the art, it is well known that there is a huge difference 

between a 'shot' as described by Abdel, and a 'single frame' as claimed Abdul 

I 

correctly defines a "shot' (or scene) consisting of a sequence of uninterrupted 
related frames," see Abstract Applicants assert that a 'sequence* as described by 
Abdul can never be a single frame as claimed. 

That is, a shot includes aU frames from when a shutter of a camera opens, until the 
shutter closes. At typical frame rate of ^0 or 60 frames per second, it would be 
impossible to perceive a shot as described by Abdel if it were to include only a 
single frame as claimed. Typically, perceivable 'shots' must last from at least 
several seconds to several minutes. This means that 'shots' typically include at least 
hundreds if not thousands of frames. 

I 

Now, Abdul determines a level of activity in a shot, and then performs the selected 
speed change special effect by adding ^ames or deleting frames to produce a 
modified shot, sec Abstract. The invention operates in each frame as it comes by. In 
the most extreme case, and a most liberal interpetation of Abdel the shot must 
include at least a first frame and a second frame, see colimm 2, 
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45 The video processing device comprises an image processor 
capable of identifying the at least one shot and determining 
a first activity level widiin at least a first portion of the at 
least one shot, wherein the first activity level indicates a rate 
of change of video content between at least a first frame and 

5(j a second frame in the at least a first portion. The image 
processor performs the selected speed change special effect 
by one of adding frames and deleting frames in the at least 
a first portion in response to the; first activity level 
determination, thereby producing a modified at least one 

55 shot. 



There is a serious problem vidth the Abdel system and method. Abdel can never 
operate in real time. This is obvious. Aljdel needs to process a shot before a new 



shot can be created. Thus, if the shot is 



a minute long, the viewer sees nothing until 



at least a minute later, when the result of his processing becomes available. 

In contrast, the invention processes the video and frame at the same time. As each 
input frame is classified, play time is allocated to each corresponding output frame, 
and the output can be viewed, as is, befj^re any more frames need to be processed, 
as in Abdel. 

Abdel is shot based. Abdel classifies shots, see col. 7, lines 33-38, below: 

The shot frames are 
examined lo determine the relative degree of movement 
between successive frames and/or across i a longer series of 
frames within the shot. I'hc shot (or a Sub-portion of rhe 
.shoi) may ihen be clas.siriecl according to the level of acliviiy 
in ihe shot (or sub-portion). , 

See also, Figures 3, 4A and 4B. Paxticurarly, see element 320 of Figure 3, element 
415 of Figure 4A, and element 460 of Figure 4B, each describing shot based 
classification. 
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In claims 2-3, the content characteristic is based on low-level features of each of the 
input frames. The low-level features are selected from a group consisting of motion 
vectors, color, texture, brightness, contrast, spectral parameters, local and global 
motion, activity, trajectory, speed, acceleration, object shape, object size, number of 
objects, number effaces, pitch, volume, tempo, and combinations thereof. The low 
level features for a frame are derived fi|om each input frame for each corresponding 
output frame in the invention. Abdel measures frame differences in shots, which is 
not what is claimed. 

In claim 6, the allocating of the play tirne is dynamically varied while processing 
the video. In claim 21, a total output video play time is allocated for an output video 
and the output frame play times are allocated so that a sum of the output frame play 
times of the plurality of output frames ^ equal to the total output video play time of 
the output video.As stated above, play time is frame based in the invention. Abdul 
increases the speed of a shot, see col. 8, 

Otux the iwhtiw oioount of movcmeai wiihia ihc shot is 
3^ cla.s.sificd a.^; high u» low, the speed of ihe shot jb ioetcased 

in reliUion to Ihe oristnal sb»i by savio^ sub-?jmp)«s of 

framiav at a high or low rate. If the shol & cbssillal a.s 

low.lisvcl J^ct^^i(y, \\k sub-sampl«: occurs al a coiipa/ativeJy 

low rate (process.4iep 42U). "lbs low siib^umple W n:.-suli» 
41 in i rclalivcly .smaller number of 6ra0i«v being siivcd »{]d a 

T«;Jaiivcly latter ouaiber of fnnie.; being dcpppeq i>r ikkivd 

frojn ihe originjii .shM. For example, if the aciMiy level is 

luw in a (or 3 suh-ponioo of the .sbot). image proctiwutr 120 

may .save one out nf tn*cry ihree fnmi^ and drop the eihor 
^, rxo frames. 

i 

Above, Abdel classifies the amount of rnovement m a shot. This is further 
described in col. 9: 
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(niftuies. The tiUc at which froixies tifst dropped ckirins tbt 
fast-forvvarrf sjjeeil cbanjio is Uien ioiiiii]]^ calailaied by 
anagopruucsiior 120 ia order {o comply mih ihc fiyc minuiv 
time ceiRiraint. (mage proceswr 120 niiiy tben Toop throi^b .< 
tlMs vkfeo dip one or raw liroes u> make lb: Hoia] iaiodificd 
vklbo clip [>iay in five mimites or less. Image {tmcdftsof 120 
*an apply difEsrunt wsighiing fcaots lo bigJi-odMiy Jevel 
ftud bw-aciiviiy ic-vci portioos of ihc video elip in order lo 
medt wirh tlic spcdfied time duratioo. 



There, Abdel must "loop through the video clip one or more times" to meet the 5 
minute constraint. This is a further indication that Abdel cannot be a real time 
process, and that Abdul is a multi-pass.operation. In contrast, the invention is a 
single pass operation, I 



In claim 7, the allocated output frame play time of each of the output frames is 
determined by sampling the input frames. In claim 8, the sampling is a down- 
sampling of the input frames. In claim 9, the sampling is an up-sampling of the 
input frames. In claim 10, up-sampled Output frames are interpolated from the input 
frames. In claim 1 1 , the sampling is a CjDmbination of down-sampling and up- 
sampling of the input frames. The inv^tion determines the allocated output frame 

play time for each output frame based o[n the classified content characteristic of 

I 

each of the input frames. Abdel never djetermines play time. Abdel determines 
activity for an entire sequence of frames in a shot and adds or drops frames based 
on the activity. Abdel can never anticipjate what is claimed. 



In claim 12, the allocated output frame play time of each of the output frames is 
detennined by an output frame rate of the output frame. In claim 1 3, the output 
frame rate is increased for selected input frames, fri claim 14, the output frame rate 
is decreased for selected input frames. The invention detennines output frame play 
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time for each output frame. Abdel never changes the frame rate for individual 
frames as claimed. Abdel determines whether the shot is to be played in fast 
forward or slow motion, and adds or drops frames of the shot to achieve a desired 
degree of motion, see col. 6 below: 

^ . 1 Since llM 

v)dco lraiis(cr rate of ihc image medium is con^jml, ihc 

degree of aiotion la tbcsc shots cm he sIo«Vcd by the 

duplicadon «iid addition of «<ilcL-t«d Braines. Owvefscly, tbe 

moiion asstKiJilcd with licsc sbols ma be iacrejsed lljn.>Mgb 
^ thd deletion or removal of Mkcied fmrncs 40 in accoidmee 

witb the priocipJes of the picseni inveailoa. 

Claimed is allocating a rate on a per frame basis. Abdel adds or drops frames to 
control a degree of motion in the shots, which has nothing to do with a rate of a 
frame. 

In claim 15, the content characteristics of each of the plurality of input frames is 
measured to determine the classification. Each frame is measured to classify the 
. frame. Abdel classifies a shot. These are totally different operations. 

In claim 19, the input video is partitioned into a plurality of segments, and the input 
video is processed on a per segment basis. Therefore, different segments may be 
processed accordmg to different selected content characteristics. The input video is 
still processed on a frame basis, which is never described by Abdel. 

In claim 21, a total output video play tinjie is allocated for an output video and the 
output frame play times are allocated so that a sum of the output frame play times 
of the plurality of output frames is equal to the total output video play time of the 
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output video. As described in col. 9, Abdel must "loop through the video clip one 
or more times" to meet the 5 minute constraint. Abdul is a multi-pass operation. In 
contrast, the invention is a single pass operation. Abdel cannot anticipate the 
invention. 

Claims 4 and 5 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Abdel in view of Mehrotra, et al. (U.S.! 6,665,423 -"Mehrotra"). 

Mehrotra generates descriptions of video objects using high-level and low-level 
time varying characteristics of the objects. The invention generates output frames 
having an output frame play time based on the selected content characteristic of 
each of the input frames, which can be Jiigh-level features of each of the input 
frames selected from a group consisting of genre, dramatic intensity, humor 
content, action level, beauty, lyricism, musical intensity, educational quality, 
profiindity, nudity, linguistic class, and jcombinations thereof. Mehrotra never 
produces output frames. Mehrotra uses features to generate descriptions of video 
objects. The invention uses features to produce output frames. Mehrotra has 
nothing to do with what is claimed. 

Claim 16 is rejected under 35 U.S.C. 103(a) as being unpatentable over Abdel in 
view of Park, et al. (U.S. 6,597,738 -"Park"). 

Park generates motion descriptors by measuring motion across an entire video or 
respective frames using various first order Statistics, see col. 22, below. 
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Ncxi. ihc motion direction dcscripior is lo describe sta- 
tistic char«terisifcs of raoticm dlrecijoB wJib reliifcai to 
raodoa <laiii for re^wive imaged or ifae wbote iiaagos in ibe 
time which isintcoded by the time d«Brip<or. aotl cwn^ised 
of a rooifon direcrioo average, central roos^eol for the 
w a\ttnfft mS standard devialion, domisaai motido diKction, 
acctanutiited inoiiott histogram and dircctioD data. 



The invention determines higher order statistics for the measured characteristics of 
each frame to detennine the classification for each frame to generate an output 
frame. Generatmg motion descriptors has nothing to do with generating an output 
frame having an output frame play time based on the selected content characteristic 

I 

of an input frame as claimed. Park's m<|)tion descriptors can never be used to make 
the invention obvious. 



All rejections have been complied with, and applicant respectfully submits that the 
application is now in condition for allo^jvance. The applicant urges the Examiner to 
contact the applicant's attorney at phone and address indicated below if assistance 
is required to move the present application to allowance. Please charge any 
shortages in fees in connection with this filing to Deposit Account 50-0749 . 



Mitsubishi Electric 
Research Laboratory, Inc. 
201 Broadway 
Cambridge MA, 02139 
(617)621-7539 



Respectfull 




Andrev/JETCurtin 
Reg. No. 48,485 
Attorney for Assignee 
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